最初受生物神经网络（BNN）启发的人工神经网络（ANN）在许多任务（例如视觉表示学习）中取得了巨大的成功。但是，由于缺乏有效的工具来链接和互为两个不同的域，并且缺乏代表的一般有效的框架，ANN和BNN中的视觉表示之间是否存在语义相关性/连接仍然很大程度上尚未探索。 BNN中的视觉语义，例如人类功能性脑网络（FBN）。为了回答这个问题，我们提出了一个新颖的计算框架，即同步激活（同步性），以基于自然主义的功能磁共振成像（NFMRI）数据来对人脑中的ANN和BNN之间的视觉表示空间和语义进行。通过这种方法，我们能够在第一次以人类脑成像得出的生物学上有意义的描述中对神经元进行注释。我们在两个公开观看的NFMRI数据集上评估了同步操作框架。该实验证明了a）FBN中视觉表示与各种卷积神经网络（CNN）模型中的视觉表示之间的显着相关性和相似性； b）CNN的视觉表示与BNN的相似性与其在图像分类任务中的性能之间的紧密关系。总体而言，我们的研究介绍了一个一般有效的范式，以融入ANN和BNNS，并为未来的研究提供新的见解，例如脑启发的人工智能。

FreeSpace检测是自动驾驶技术的重要组成部分，并且在轨迹计划中起着重要作用。在过去的十年中，已证明基于深度学习的自由空间检测方法可行。但是，这些努力集中在城市道路环境上，由于缺乏越野基准，很少有针对越野自由空间检测专门设计的深度学习方法。在本文中，我们介绍了ORFD数据集，据我们所知，该数据集是第一个越野自由空间检测数据集。数据集收集在不同的场景（林地，农田，草地和乡村），不同的天气条件（阳光，多雨，雾气和雪地）以及不同的光线条件（明亮的光线，日光，暮光，黑暗）中，完全包含12,198 LIDAR点云和RGB图像对与可穿越的区域，不可传输区域和无法达到的区域进行了详细注释。我们提出了一个名为Off-NET的新型网络，该网络将变压器体系结构统一以汇总本地和全球信息，以满足大型接收领域的自由空间检测任务的要求。我们还向动态融合激光雷达和RGB图像信息提出了交叉注意，以进行准确的越野自由空间检测。数据集和代码可公开可用athttps：//github.com/chaytonmin/off-net。

快捷方式学习对深度学习模型很常见，但导致了退化的特征表示形式，因此危害了该模型的可推广性和解释性。但是，在广泛使用的视觉变压器框架中的快捷方式学习在很大程度上是未知的。同时，引入特定领域的知识是纠正捷径的主要方法，捷径为背景相关因素。例如，在医学成像领域中，放射科医生的眼睛凝视数据是一种有效的人类视觉先验知识，具有指导深度学习模型的巨大潜力，可以专注于有意义的前景区域。但是，获得眼睛凝视数据是时必的，劳动密集型的，有时甚至是不切实际的。在这项工作中，我们提出了一种新颖而有效的显着性视觉变压器（SGT）模型，以在没有眼神数据的情况下在VIT中纠正快捷方式学习。具体而言，采用计算视觉显着性模型来预测输入图像样本的显着性图。然后，显着图用于散布最有用的图像贴片。在拟议的中士中，图像贴片之间的自我注意力仅集中于蒸馏的信息。考虑到这种蒸馏操作可能会导致全局信息丢失，我们在最后一个编码器层中进一步介绍了一个残留的连接，该连接捕获了所有图像贴片中的自我注意力。四个独立公共数据集的实验结果表明，我们的SGT框架可以有效地学习和利用人类的先验知识，而无需眼睛凝视数据，并且比基线更好。同时，它成功地纠正了有害的快捷方式学习并显着提高了VIT模型的解释性，证明了传递人类先验知识在纠正快捷方式学习方面传递人类先验知识的承诺

This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.

We aim to bridge the gap between our common-sense few-sample human learning and large-data machine learning. We derive a theory of human-like few-shot learning from von-Neuman-Landauer's principle. modelling human learning is difficult as how people learn varies from one to another. Under commonly accepted definitions, we prove that all human or animal few-shot learning, and major models including Free Energy Principle and Bayesian Program Learning that model such learning, approximate our theory, under Church-Turing thesis. We find that deep generative model like variational autoencoder (VAE) can be used to approximate our theory and perform significantly better than baseline models including deep neural networks, for image recognition, low resource language processing, and character recognition.

Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, the ability to learn from limited labeled data and to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of vocabulary-informed learning to alleviate the above mentioned challenges and address problems of supervised, zero-shot, generalized zero-shot and open set recognition using a unified framework. Specifically, we propose a weighted maximum margin framework for semantic manifold-based recognition that incorporates distance constraints from (both supervised and unsupervised) vocabulary atoms. Distance constraints ensure that labeled samples are projected closer to their correct prototypes, in the embedding space, than to others. We illustrate that resulting model shows improvements in supervised, zero-shot, generalized zero-shot, and large open set recognition, with up to 310K class vocabulary on Animal with Attributes and ImageNet datasets.

We consider infinite horizon Markov decision processes (MDPs) with fast-slow structure, meaning that certain parts of the state space move "fast" (and in a sense, are more influential) while other parts transition more "slowly." Such structure is common in real-world problems where sequential decisions need to be made at high frequencies, yet information that varies at a slower timescale also influences the optimal policy. Examples include: (1) service allocation for a multi-class queue with (slowly varying) stochastic costs, (2) a restless multi-armed bandit with an environmental state, and (3) energy demand response, where both day-ahead and real-time prices play a role in the firm's revenue. Models that fully capture these problems often result in MDPs with large state spaces and large effective time horizons (due to frequent decisions), rendering them computationally intractable. We propose an approximate dynamic programming algorithmic framework based on the idea of "freezing" the slow states, solving a set of simpler finite-horizon MDPs (the lower-level MDPs), and applying value iteration (VI) to an auxiliary MDP that transitions on a slower timescale (the upper-level MDP). We also extend the technique to a function approximation setting, where a feature-based linear architecture is used. On the theoretical side, we analyze the regret incurred by each variant of our frozen-state approach. Finally, we give empirical evidence that the frozen-state approach generates effective policies using just a fraction of the computational cost, while illustrating that simply omitting slow states from the decision modeling is often not a viable heuristic.

We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io

Reinforcement Learning (RL) is currently one of the most commonly used techniques for traffic signal control (TSC), which can adaptively adjusted traffic signal phase and duration according to real-time traffic data. However, a fully centralized RL approach is beset with difficulties in a multi-network scenario because of exponential growth in state-action space with increasing intersections. Multi-agent reinforcement learning (MARL) can overcome the high-dimension problem by employing the global control of each local RL agent, but it also brings new challenges, such as the failure of convergence caused by the non-stationary Markov Decision Process (MDP). In this paper, we introduce an off-policy nash deep Q-Network (OPNDQN) algorithm, which mitigates the weakness of both fully centralized and MARL approaches. The OPNDQN algorithm solves the problem that traditional algorithms cannot be used in large state-action space traffic models by utilizing a fictitious game approach at each iteration to find the nash equilibrium among neighboring intersections, from which no intersection has incentive to unilaterally deviate. One of main advantages of OPNDQN is to mitigate the non-stationarity of multi-agent Markov process because it considers the mutual influence among neighboring intersections by sharing their actions. On the other hand, for training a large traffic network, the convergence rate of OPNDQN is higher than that of existing MARL approaches because it does not incorporate all state information of each agent. We conduct an extensive experiments by using Simulation of Urban MObility simulator (SUMO), and show the dominant superiority of OPNDQN over several existing MARL approaches in terms of average queue length, episode training reward and average waiting time.

The growing interest in intelligent services and privacy protection for mobile devices has given rise to the widespread application of federated learning in Multi-access Edge Computing (MEC). Diverse user behaviors call for personalized services with heterogeneous Machine Learning (ML) models on different devices. Federated Multi-task Learning (FMTL) is proposed to train related but personalized ML models for different devices, whereas previous works suffer from excessive communication overhead during training and neglect the model heterogeneity among devices in MEC. Introducing knowledge distillation into FMTL can simultaneously enable efficient communication and model heterogeneity among clients, whereas existing methods rely on a public dataset, which is impractical in reality. To tackle this dilemma, Federated MultI-task Distillation for Multi-access Edge CompuTing (FedICT) is proposed. FedICT direct local-global knowledge aloof during bi-directional distillation processes between clients and the server, aiming to enable multi-task clients while alleviating client drift derived from divergent optimization directions of client-side local models. Specifically, FedICT includes Federated Prior Knowledge Distillation (FPKD) and Local Knowledge Adjustment (LKA). FPKD is proposed to reinforce the clients' fitting of local data by introducing prior knowledge of local data distributions. Moreover, LKA is proposed to correct the distillation loss of the server, making the transferred local knowledge better match the generalized representation. Experiments on three datasets show that FedICT significantly outperforms all compared benchmarks in various data heterogeneous and model architecture settings, achieving improved accuracy with less than 1.2% training communication overhead compared with FedAvg and no more than 75% training communication round compared with FedGKT.